Probability Inequalities for Kernel Embeddings in Sampling without Replacement

The kernel embedding of distributions is a popular machine learning technique to manipulate probability distributions and is an integral part of numerous applications. Its empirical counterpart is an estimate from a finite set of samples from the distribution under consideration. However, for large-scale learning problems the empirical kernel embedding becomes infeasible to compute and approximate, constant time solutions are necessary. One can use a random subset of smaller size as a proxy for the exhaustive set of samples to calculate the empirical kernel embedding which is known as sampling without replacement. In this work we generalize the results of Serfling (1974) to quantify the difference between the full empirical kernel embedding and the one estimated from random subsets. Furthermore, we derive probability inequalities for Banach space valued martingales in the setting of sampling without replacement.